Synthesizing Optimal Domain Models for Educational Applications

Many applications in educational technology, from student modeling to problem generation, are built on a formal model of the operational knowledge for a given domain. These domain models consist of rules that experts apply to solve problems in the domain; for example, factoring, ax + bx → (a + b)x , is one such rule for K-12 algebra. In practice, domain models are handcrafted at great expense, so applications rely on a single generic model. But many models can capture the operational knowledge for a domain, and they differ in how well they meet specific educational objectives (such as maximizing problem-solving efficiency for advanced users or minimizing cognitive load for novices). Rapid creation of custom domain models is thus a key challenge in the development of personalized educational tools that adapt to their user’s needs. This paper presents RuleSy, a new framework for computer-aided authoring of optimal domain models for educational applications. RuleSy takes as input a set of example problems (e.g., x + 1 = 2), a set of basic axiom rules for solving these problems (e.g., factoring), and a function expressing the desired educational objective. Given these inputs, it first synthesizes a set of tactic rules (e.g., combining like terms) that integrate multiple axioms into advanced problem-solving strategies. The axioms and tactics are then searched for a domain model that optimizes the objective. RuleSy is based on new algorithms for mining tactic specifications from examples and axioms, synthesizing tactic rules from these specifications, and selecting an optimal domain model from the axioms and tactics. We evaluate RuleSy on the domain of K-12 algebra, finding that it recovers textbook tactics and domain models, discovers new tactics and models, and outperforms a prior tool for this domain by orders of magnitude. But RuleSy generalizes beyond K-12 algebra: we also use it to (re)discover proof tactics for propositional logic, demonstrating its potential to aid in the development of custom models for a variety of educational domains.